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Summary 


Problem  and  Background 

The  challenges  facing  Antisubmarine  Warfare  (ASW)  training  in  the  closing  years  of 
this  decade  are  greater  than  at  any  time  since  the  early  days  of  World  War  H  Conditions 
since  the  end  of  the  Cold  War,  and  those  expected  throughout  the  next  decade,  impose 
additional  complexity  on  maintaining  our  ASW  superiority.  Russian  nuclear  submarine 
technology  continues  to  improve  and  advanced  submarines  continue  to  be  built  and 
delivered  to  their  fleet.  Concurrently,  the  proliferation  of  improved  diesel  submarine 
technology  to  many  Third  World  nations  requires  that  our  ASW  forces  also  be  capable  of 
conducting  operations  in  the  vastly  different  littoral  regions. 

When  coupled  with  dramatic  reductions  in  ASW  training  resources,  including  at-sea 
training,  this  historic  change  compels  the  development  of  training  for  skills  learned 
previously  on-the-job  and  for  skills  required  in  new  environments.  The  training  challenge 
is  two-fold:  (1)  retaining  the  capability  to  detect  and  prosecute  nuclear  submarines,  and 
(2)  expanding  our  current  capability  against  diesel  submarines  of  the  Third  World. 

The  Interactive  Multisensor  Analysis  Training  (MAT)  system  was  developed  to 
address  post  Cold  War  ASW  training  requirements.  Specifically,  MAT  is  designed  to 
teach  the  complex  conceptual  knowledge  and  cognitive  and  procedural  skills  required  to 
reason  about  the  interrelationships  among  the  operating  modes  of  target  submarines,  the 
environmental  variables  that  affect  sound  transmission,  and  the  sensor  systems  used  for 
detection  and  localization.  In  addition,  IMAT  provides  extensive  training  on  submarine 
acoustics  and  interpretation  of  sensor  system  data.  The  MAT  approach  to  training  is 
based  on  recent  advances  in  cognitive  psychology  and  instructional  technology.  It 
combines  cognitive  analytic  and  curriculum  design  technology  with  advanced  computer- 
based  graphics  and  programming  technology  to  deliver  state-of-the-art  training. 

Objective 

The  objective  of  this  effort  was  to  evaluate  the  MAT  system  based  Airborne 
Acoustic  Mission  Course  (AAMC)  at  the  Aviation  Systems  Warfare  Operator  (AW)  “A” 
School. 

Method 

There  were  two  components  to  the  evaluation.  First,  a  small  group  of  AAMC  trainees 
(N  =  11)  were  compared  on  a  knowledge  test  with  a  small  number  of  graduates  of  the 
Common  Core  Acoustic  Analysis  (CCAA)  (N  =  7).  The  AAMC  replaced  the  CCAA 
course.  Second,  a  larger  group  of  AAMC  trainees  (. N  =  35)  were  evaluated  on  pretest- 
posttest  knowledge  test  performance  and  for  accelerated  expertise  on  acoustic  mission 
tasks.  Accelerated  expertise  was  evaluated  by  comparing  trainee  performance  on  selected 
AAMC  scenario  tasks  to  a  list  of  skills  deemed  essential  to  mission  success  by  six  fleet 
subject-matter  experts  at  supervisory  levels.  The  same  subject-matter  experts  rated  those 
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skills  by  assessing  when  typical  operators  would  be  able  to  perform  those  skills 
competently,  with  minimal  supervision. 

Results 

AAMC  trainees  performed  significantly  better  than  CCAA  trainees  on  the  knowledge 
tests.  AAMC  trainees  also  showed  significant  improvement  from  the  AAMC  knowledge 
pretests  to  the  posttests.  Finally,  AAMC  trainee  performance  on  the  scenario  tasks 
showed  clear  evidence  of  accelerated  expertise. 

Conclusions 

Two  conclusions  can  be  drawn  from  this  study.  First,  the  IMAT  approach  to  training, 
which  is  based  on  computer-generated  dynamic  displays  and  instructional  design  and 
delivery  strategies  designed  to  enhance  cognition,  produced  substantial  gains  in  subject 
matter  knowledge.  Second,  performance  on  the  AAMC  scenario  tasks  support  the 
conclusion  that  AAMC  trainees  experienced  accelerated  expertise.  This  finding  indicates 
that  the  IMAT-based  AAMC  should  be  capable  of  overcoming  the  lack  of  practice 
opportunities  available  in  today’s  typical  fleet  operations.  Overall,  the  results  show  that 
IMAT  is  a  highly  effective  training  system  that  offers  a  viable  solution  for  many  of  the 
training  requirements  and  challenges  faced  in  the  post  Cold  War  world. 
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Introduction 


Problem  and  Background 

The  challenges  facing  Antisubmarine  Warfare  (ASW)  training  in  the  closing  years  of 
this  decade  are  greater  than  at  any  time  since  the  early  days  of  World  War  II.  During  the 
Cold  War,  ASW  mission  requirements  (and  the  training  designed  to  support  those 
requirements)  were  driven  by  the  need  to  combat  Soviet  nuclear  submarines  in  open 
ocean  environments.  The  prevailing  ASW  strategy  was  designed  to  detect  and  prosecute 
enemy  submarines  at  long  ranges,  and  operations  were  most  often  conducted  by  single 
ASW  units  with  minimal  coordination  with  outside  assets.  The  relatively  benign  deep 
ocean  environment  and  nearly  exclusive  focus  on  the  Soviet  threat  resulted  in  the 
development  of  effective  and  well  practiced  sensor  and  weapons  tactics.  ASW  training 
mirrored  that  relatively  narrow  focus,  and,  combined  with  frequent  real-world  encounters, 
was  quite  effective  in  producing  competent  ASW  sensor  operators. 

Conditions  since  the  end  of  the  Cold  War,  and  those  expected  throughout  the  next 
decade,  impose  additional  complexity  on  maintaining  our  ASW  superiority.  Russian 
nuclear  submarine  technology  continues  to  improve,  and  advanced  submarines  continue 
to  be  built  and  delivered  to  their  fleet.  Concurrently,  the  proliferation  of  improved  diesel 
submarine  technology  to  many  Third  World  nations  requires  that  our  ASW  forces  also  be 
capable  of  conducting  operations  in  the  vastly  different  littoral  regions. 

Littoral  environments  introduce  added  difficulty  in  optimally  employing  onboard 
sensors  and  weapons  tactics.  Coastal  areas  with  shallow  water,  complex  bathymetry  and 
bottom  topography,  heavy  shipping,  and  highly  variable  environmental  conditions 
impose  significant  restrictions  on  traditional  deep  water  tactics.  The  diesel  submarine 
operating  in  home  waters  has  the  added  advantage  of  familiarity  with  environmental  and 
topographical  anomalies,  slow  speed  operation,  and  support  from  own  nation  defense 
systems. 

When  coupled  with  dramatic  reductions  in  ASW  training  resources,  including  at-sea 
training,  this  historic  change  compels  the  development  of  different  approaches  to  training 
for  sensor  operators.  The  training  challenge  has  thus  become  two-fold:  (1)  retaining  the 
Cold  War  capability  to  detect  and  prosecute  nuclear  submarines  and  (2)  expanding  our 
current  capability  against  diesel  submarines  of  the  Third  World. 

Traditional  ASW  Training  for  Sensor  Operators 

In  the  past,  ASW  training  has  been  based  on  a  balance  between  schoolhouse  and 
operational  training.  To  become  journey  level  operators,  trainees  learned  the  basics  in 
schoolhouse  training  and  then  received  extensive  on-the-job  training  through  supervised 
practice  by  experienced  fleet  crews  at-sea.  The  schoolhouse  training  focused  on 
memorizing  facts,  procedures,  and  large  databases  of  threat  intelligence  parameters. 
Trainees  were  not  taught  to  think  about  and  relate  the  underlying  physics  of  threats  to  the 
environment  and  to  the  sensor  systems.  This  approach  resulted  in  graduates  who  could 
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perform  procedures,  but  who  had  difficulty  in  applying  knowledge  and  principles  to  solve 
problems  in  operational  situations.  Great  reliance  was  placed  on  a  substantial  amount  of 
at-sea  experience  to  transfer  the  knowledge  and  skill  gained  in  formal  training  to 
operational  competency. 

Post  Cold  War  ASW  Training  Requirements 

There  is  now  a  broad  consensus  across  all  four  ASW  training  communities  that  the 
combined  change  in  threat,  environment,  and  operating  circumstances  requires  a  new 
training  approach. 

Current  fleet  ASW  practitioners  receive  little  at-sea  operational  practice  against  non- 
cooperative  submarines.  As  a  result,  an  operator’s  ability  to  detect,  classify,  and  track 
submarine  targets  will  be  directly  related  to  the  quality  of  initial  training,  and  to  how  well 
those  skills  are  maintained  and  increased  throughout  a  career.  To  achieve  this  level  of 
competence,  the  limited  hours  available  for  both  schoolhouse  and  at-sea  training  must  be 
used  to  provide  both  a  solid  conceptual  understanding  of  the  complex  tasks  that  operators 
must  perform  as  well  as  opportunities  to  practice  those  tasks  under  varying  real-world 
conditions. 


Achieving  these  goals  will  require  substantive  changes  in  both  schoolhouse  and 
operational  training.  ASW  training  must  see  dramatic  modifications  to  passive  acoustic 
analysis  training  and  substantial  expansion  of  training  in  active  sonar,  radar,  and 
electromagnetic  sensor  systems  to  enable  operators  to  contribute  effectively  in  a  multi¬ 
sensor  approach  to  submarine  prosecution.  Further,  the  complexity  of  the  acoustic 
environment  in  the  littoral  regions  requires  a  substantial  increase  in  knowledge  of  the 
effects  of  ocean  bathymetry  on  acoustic  energy  transmission  and  how  that,  in  tum,  affects 
sensor  selection  and  placement.  These  training  requirements  can  only  be  met  by  applying 
principle-based  training  technologies  that  can  provide  conceptual  knowledge  and  high 
fidelity  experience  to  offset  the  lack  of  at-sea  practice. 

The  Interactive  Multisensor  Analysis  Training  (IMAT)  System 

Overview:  The  IMAT  system  (Ellis  &  Parchman,  1994;  Wetzel-Smith,  Ellis, 
Reynolds,  &  Wulfeck,  1995)  was  developed  to  address  post  Cold  War  ASW  training 
requirements.  Specifically,  IMAT  is  a  classroom  based  approach  to  training  that  is 
designed  to  teach  the  complex  conceptual  knowledge  and  cognitive  and  procedural  skills 
required  to  reason  about  the  interrelationships  among  the  operating  modes  of  target 
submarines,  the  environmental  variables  that  affect  sound  transmission,  and  the  sensor 
systems  used  for  detection  and  localization.  In  addition,  IMAT  provides  extensive 
training  on  submarine  acoustics  and  interpretation  of  sensor  system  data. 

The  IMAT  approach  to  training  is  based  on  recent  advances  in  cognitive  psychology 
and  instructional  technology.  It  combines  cognitive  analytic  and  instructional  design 
technology  with  advanced  computer-based  graphics  and  programming  technology.  The 
result  is  a  dynamic  graphical  interface  integrated  with  state-of-the-art  instructor  and 
trainee  guides  that  provides  the  traditional  classroom  instructor  with  a  capability  to 
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effectively  teach  complex  cognitive  concepts  and  skills.  In  the  past,  achieving  this 
capability  has  been  hampered  by  limitations  in  cognitive  task  analysis,  and  particularly  by 
limitations  in  cognitive  models.  In  addition,  limitations  in  computer  capabilities  have 
precluded  the  development  of  cause  and  effect  representations  of  highly  complex,  multi¬ 
modal  tasks  that  are  required  of  expert  practitioners.  Recent  developments  in  cognitively- 
based  training  design  have  demonstrated  that  models  of  physical  phenomena  can  be 
integrated  with  high  resolution  graphics  to  demonstrate  the  interactive  relationships  of 
threat,  environment,  and  system  for  operator  training.  The  IMAT  system  extends  these 
technologies  to  the  traditional  classroom  environment,  with  specific  emphasis  on 
cognitive  design  models  that  account  for  knowledge  structure  interrelationships.  The 
result  is  cause  and  effect  training  that  incorporates  visualization  of  interactive  spatial 
relationships  among  operators,  sensors,  and  other  platforms. 

Specifically,  for  sensor  operators,  IMAT  apprentice  training  is  presented  in  a  mission 
context  with  a  substantial  emphasis  on  the  interactive  relationship  among  environmental 
factors,  threat  behavior,  and  sensor  system  capabilities  and  constraints.  Because 
apprentice  sensor  operators  have  limited  backgrounds  in  submarine  operations,  the 
physics  of  energy  transmission,  and  in  operating  complex  sensor  systems,  establishing  a 
good  conceptual  understanding  of  the  complicated  real  world  interactions  is  the  essential 
foundation  for  learning  effective  sensor  operation.  Operator  trainees  are  taught  to 
understand  that  the  enemy  submarine’s  mission  will  largely  dictate  its  operating  mode 
(including  course,  speed,  and  depth  of  operation)  and  that  the  operating  mode  of  the 
submarine  defines  its  vulnerabilities  to  onboard  acoustic  and  electromagnetic  sensors. 
They  are  further  taught  how  the  relative  complexity  of  the  ocean  environment  will  impact 
detection  ranges,  search  rates,  and  contact  duration.  Trainees  learn  to  reason  through 
these  interactions  in  a  cause  and  effect  learning  process.  Multi-dimensional  interactions 
are  displayed  visually,  and  the  IMAT  instruction  provides  qualitative  explanations  for  the 
interactions  that  occur.  IMAT  presents  demonstrations  of  varying  outcomes  based  upon 
changes  in  the  threat  or  environment  to  promote  the  development  of  the  principle-based 
knowledge  critical  for  adaptations  to  the  variations  presented  in  real  world  situations. 
Finally,  tests  of  trainee  proficiency  for  IMAT  training  include  questions  that  require 
problem  solving  and  understanding  causal  relationships  in  addition  to  the  traditional  fact 
and  procedural  knowledge  items. 

Research  Background 

The  IMAT  system  integrates  several  areas  of  research  on  cognition  and  instruction, 
including,  graphical  techniques  to  promote  visualization  of  invisible  phenomena  in 
science  teaching,  elaborated  explanations,  contextualized  or  anchored  instruction,  and 
instructional  sequencing.  The  following  sections  briefly  summarize  portions  of  this  work. 

Scientific  Visualization:  Scientific  visualization  has  traditionally  been  used  by 
scientists  to  explore  phenomena  and  to  communicate  with  other  scientists  (Bryson,  1994). 
When  used  for  presentations,  researchers  select  data  sets,  transforms  them  and  then  turn 
them  over  to  specialized  graphic  artists  to  develop  images  and  animation.  However,  the 
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end  products  of  this  process  have  not  been  designed  for  lay  persons  or  trainees.  IMAT 
aims  to  bring  this  technology  into  specialized  technical  training. 

Research  support  for  scientific  visualization  as  a  training  strategy  comes  from  the 
literature  on  instructional  media.  Both  static  and  dynamic  graphic  displays  have  been 
shown  to  facilitate  teaching  of  scientific  concepts  (Baek  &  Layne,  1988;  Dwyer,  1972; 
Gropper,  1966;  Lumsdaine,  Sulzer,  &  Kopstein,  1961;  Rieber,  1990;  Rigney  &  Lutz! 
1975;  Park  &  Gittelman,  1992,  Wetzel,  Radtke,  &  Stem,  1994).  Levie  and  Lentz  (1982), 
in  a  meta-analysis  of  illustrated  text  studies,  concluded  that  learning  and  retention  is 
facilitated  by  illustrations,  if  the  illustrations  are  directly  related  to  the  text.  Park  and 
Gittelman  (1992)  found  that  subjects  trained  with  dynamic  graphics  performed  better  on 
electronic  troubleshooting  problems  than  those  trained  with  static  displays.  White  (1984) 
used  animated  computer  graphics  to  successfully  teach  the  basic  principles  of  Newtonian 
laws  of  motion  and  force.  IMAT  employs  a  computer  based  graphical  interface  to 
conceptual  models  of  real  world  phenomena  to  deliver  both  static  and  dynamic  graphics 
in  a  traditional  classroom  environment. 

Elaborated  Explanations:  Providing  students  with  elaborated  explanations, 
analogies,  etc.,  about  how  and  why  systems,  events,  and  phenomena  are  structured  and 
function  has  been  shown  to  facilitate  learning  and  retention.  Research  on  learning  skills 
and  on  learning  from  text  has  shown  that  elaborated  explanations  enhance  the  students’ 
mental  models  and  increase  retention  (Mayer,  1989;  Konoske  &  Ellis,  1991;  Smith  & 
Goodman,  1982).  In  a  series  of  studies  of  learning  from  scientific  text,  Mayer  (1989) 
found  that  providing  students  with  a  conceptual  model  increased  learning,  retention  and 
transfer.  The  conceptual  models  in  his  instruction  used  both  text  and  diagrams  to 
highlight  major  objects  and  actions  and  the  causal  relations  among  them.  That  is,  the 
models  focused  on  how  and  why  systems  work.  Smith  and  Goodman  (1982)  studied  the 
effects  of  providing  elaborated  instructions  on  learning  and  performing  a  procedural 
assembly  task  and  found  that  instructions  containing  functional  information  resulted  in 
fewer  errors.  Swezey,  Perez,  and  Allen  (1991),  in  a  study  on  transfer  of  electromechanical 
troubleshooting  skill,  found  that  some  level  of  generic  structure  and  functional  knowledge 
is  required  for  cross  domain  transfer.  The  IMAT  system  uses  elaborated  explanations 
throughout  the  instruction  to  (1)  clarify  complex  relationships  such  as  those  among  water 
temperature,  pressure,  depth,  and  salinity;  (2)  provide  comprehensive  feedback  for 
practice  exercises;  and  (3)  describe  graphically  displayed  examples. 

Contextualized  Instruction  and  Instructional  Sequencing:  Contextualized  or  job 
oriented  instruction  has  been  found  to  be  more  effective  in  learning,  retention,  and 
performance  than  topic  oriented  instruction  (Semb  &  Ellis,  1994;  Johnson,  1951; 
Goffard,  Heimstra,  Beecroft  &  Oppenshaw,  1960;  Shoemaker,  1960;  Steinemann! 
Hamgan,  &  VanMatre,  1967;  Cognition  &  Technology  Group  at  Vanderbilt,  1990; 
Collins,  Brown,  &  Newman,  1989).  Further,  within  a  job  context,  mental  model 
development  is  facilitated  by  teaching  students  to  reason  about  events  and  phenomena 
that  involve  several  interrelated  variables. 
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Proper  sequencing  may  play  an  important  role  in  cognitive  skill  development.  While 
early  research  on  sequencing  showed  that  with  simplified  or  isolated  tasks,  different 
sequences  of  instructional  events  made  little  difference,  more  recent  research  and  theory 
suggest  that,  for  complex  tasks,  sequencing  strategies  may  have  significant  effects.  For 
example,  Reigeluth  and  Stein  (1983)  argue  that  beginning  instruction  with  a  condensed 
“holistic”  overview  of  a  task  domain  leads  to  better  learning  than  more  traditional 
sequences,  which  teach  isolated  topics  first  and  integrate  them  later.  More  recently, 
extreme  “constructivist”  approaches  to  instruction  (e.g.,  Duffy  &  Jonassen,  1991)  argue 
that  learners  should  “sink-or-swim”  in  a  fully  elaborated  domain.  Merrill,  Li,  &  Jones 
(1990)  also  argue  for  a  holistic  approach  to  teaching  complex  domains,  but  include 
moderate  structure  and  sequencing  recommendations  in  their  approach.  Drawing  from 
Reigeluth  and  Stein  (1983),  IMAT  begins  with  a  simplified  overview  of  target, 
environment,  and  sensor  system  relationships  in  the  context  of  the  jobs  and  tasks 
performed  by  operators  and  tacticians.  This  context  is  revisited  throughout  the  IMAT 
course  to  reinforce  the  reality  that  trainees  are  learning  to  do  a  job,  not  memorize  a  list  of 
topically  related  facts. 

AW  “A”  School  — The  Airborne  Acoustic  Mission  Course  (AAMC) 

In  each  of  these  areas,  little  experimental  work  has  been  done  on  how  well  the 
findings  generalize  to  instruction  delivered  using  simulation  and  graphical-interface- 
based  training  technologies.  Furthermore,  there  are  almost  no  larger  efforts  which 
evaluate  the  integration  of  these  approaches  into  an  overall  strategy.  The  current  effort 
tests  the  hypothesis  that  the  IMAT  system,  which  represents  an  integrated  combination  of 
these  approaches,  offers  a  potent  learning  environment  for  promoting  acquisition  of  the 
complex  knowledge  and  skills  involved  in  sensor-system  operation.  Specifically,  this 
report  documents  the  implementation  and  evaluation  of  the  IMAT  AAMC  in  the  AW  “A” 
School  at  NAS  Pensacola,  FL.  The  AAMC  replaced  the  five  and  one  half  week  Common 
Core  Acoustic  Analysis  (CCAA)  course  that  was  taught  at  the  Fleet  Aviation  Specialized 
Operations  Training  Groups  Pacific  (North  Island  Naval  Air  Station,  San  Diego,  CA)  and 
Atlantic  (Naval  Air  Station,  Jacksonville,  FL).  The  CCAA  training  used  a  topical 
approach  to  analysis  methods  and  a  linear  progression  through  a  large  number  of 
submarine  examples.  Evaluation  was  based  on  a  written  knowledge  test  (multiple  choice) 
and  analysis  of  static  LOFAR  displays.  Neither  the  written  nor  performance  exams 
presented  the  material  in  an  operational  context.  None  of  the  instruction  was  focused  on 
individual  skills  in  target  or  environmental  research,  resource  use,  or  personal  preparation 
for  detecting,  localizing,  and  tracking  submarine  contacts. 

In  contrast,  the  AAMC  builds  on  previous  “A”  School  training  by  presenting 
opportunities  to  leam  and  practice  AW  core  knowledge  and  skills  in  a  mission  context. 
The  course  is  based  on  nine  scenarios.  Each  centers  on  operations  against  a  specific 
submarine  in  a  well  defined  environment.  Examples  of  both  targets  and  operating  areas 
were  chosen  to  represent  a  wide  range  of  missions,  submarine  engineering  sophistication, 
oceanographic  characteristics,  and  sensor  capabilities.  The  instructional  design  presents 
the  first  mission  scenarios  in  highly  structured  ways  with  considerable  instructor 
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mediation.  As  learners  progress  through  the  course,  they  are  expected  to  perform  more 
autonomously  until  the  final  scenario,  which  is  almost  entirely  unmediated. 

In  addition  to  the  application  of  oceanographic  concepts  and  acoustic  analysis  skills, 
the  AAMC  places  considerable  emphasis  on  personal  preparation  for  mission 
performance.  In  the  same  scenario  concept,  learners  use  fleet  publications  and  other 
materials  to  research  both  the  target  and  environment.  These  preparatory  skills  are 
directly  transferable  to  squadron  duties  and  enable  the  users  to  “ramp  up”  after  periods  of 
ASW  inactivity. 

In  summary,  the  IMAT  based  AAMC  represents  a  significant  change  from  the  CCAA 
instruction.  The  course  length  was  increased  from  212  to  230  hours,  and  there  were 
extensive  changes  in  both  the  instructor  and  trainee  guides,  which  were  redesigned  using 
IMAT  criteria  to  provide  both  elaborated  explanations  and  more  contextualization. 
Furthermore,  the  IMAT  graphical  interface  was  used  to  provide  the  classroom  instructor 
with  both  static  and  dynamic  displays  of  important  concepts  and  relationships,  including, 
LOFAR  displays,  submarine  operating  characteristics,  submarine  equipment  acoustic 
profiles,  and  target  analysis  and  classification.  All  of  these  changes  resulted  in  a  course 
that  emphasized  learning  cognitive  skills  and  complex  relationships  instead  of 
memorizing  factual  information,  which  was  the  focus  of  the  CCAA  course. 

Objective 


The  objective  of  this  effort  is  to  evaluate  the  implementation  of  the  AAMC  with 
respect  to  knowledge  acquisition  and  accelerated  expertise  on  acoustic  mission  tasks. 

Method 


Design 

There  were  two  components  to  the  evaluation.  First,  a  small  group  of  AAMC  trainees 
(N  =  11)  were  compared  on  a  knowledge  test  with  a  small  number  of  graduates  of  the 
Common  Core  Acoustic  Analysis  (CCAA)  (N  =  7).  Second,  a  larger  group  of  AAMC 
trainees  (N  =  35)  were  evaluated  on  (1)  knowledge  acquisition  during  the  course,  which 
was  assessed  by  a  pretest  and  posttest  comparison  on  the  course  knowledge  tests  and  (2) 
accelerated  expertise  on  ASW  mission  skills. 

To  assess  accelerated  expertise,  six  senior  enlisted  AW  subject  matter  experts 
reviewed  a  profile  of  skills  and  tasks  taught  in  the  scenario  exercises  of  the  AAMC.  First, 
they  were  asked  to  rate  each  skill  for  importance  to  mission  accomplishment  and  then 
were  asked  to  estimate  the  length  of  time  required  for  apprentice  AWs  to  become 
proficient  in  each  skill.  AAMC  trainees  were  scored  on  each  skill  during  a  course 
scenario  exercise.  The  time  to  proficiency  ratings  were  then  used  to  estimate  the  degree 
of  accelerated  expertise  for  each  skill. 
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Subjects 


For  the  comparison  between  the  AAMC  and  CCAA  course,  the  subjects  were  11 
graduates  from  the  AAMC  and  7  graduates  from  the  CCAA  course.  For  the  pretest  and 
posttest  and  accelerated  expertise  comparisons,  the  subjects  were  35  trainees  from  two 
AAMC  classes.  The  six  subject  matter  experts  responding  to  the  mission  skills  survey 
were  all  experienced  air  crew  supervisors  with  multiple  sea  tours  in  fixed-wing  aircraft 
(VP  and  VS).  Two  of  the  subject  matter  experts  had  helicopter  experience.  One  subject 
matter  expert  was  an  AWCS,  four  were  AWCs,  and  one  was  an  AW1. 

Mission  Skills  Survey 

The  six  subject  matter  experts  were  given  a  list  of  skills  taught  in  the  AAMC  course 
and  were  asked  to  rate  each  skill  for  mission  importance  and  time  to  proficiency.  For 
mission  importance  the  experts  were  asked,  “To  what  extent  are  each  of  the  following 
skills  important  for  mission  success?”  They  rated  each  skill  on  a  four-point  scale:  (1) 
Unnecessary;  (2)  Nice  to  Have;  (3)  Probably  Required;  and  (4)  Absolutely  Required. 
Table  1  displays  the  list  of  skills.  The  skills  listed  in  Table  1  are  the  skills  taught  in  the 
laboratory  based  scenario  exercise  portions  of  the  AAMC.  A  number  of  additional  skills 
are  taught  in  the  classroom  portion  of  the  course  and  are  not  listed  in  Table  1. 

Table  1 

AAMC  Course  Skills  and  Behaviors 


Skill  or  Behavior 

Call  initial  contact. 

Select  appropriate  signature  components  for  tracking. 

Record  time,  frequency,  sonobuoy,  and  appropriate  notes  for  all  acoustic  signature 

components  of  interest. 

Recognize  and  report  intermittent  and  transient  acoustic  sources. 

Recognize  and  report  speed-related  components. 

Recognize  and  report  closest  point  of  approach,  from  weak  indicators  to  obvious 

ones. 

Calculate  speed  from  propulsion-related  components. 

Report  speed  changes  in  a  timely  manner. 

Report  and  record  contact  and  major  component  losses. 

Prepare  an  off-station  report  including  classification  of  target  of  interest  with 

justification  based  on  acoustic  signature. 

For  time  to  proficiency  the  experts  were  asked  “In  your  experience,  at  which  point 
would  you  expect  an  AW  “A”  School  graduate  to  be  fully  proficient  in  these  skills 
(through  a  combination  of  training  and  experience)?  They  answered  this  question  by 
rating  each  skill  on  a  6  point  scale  representing  events  in  a  career  progression:  (1)  Upon 
Reporting,  (2)  Before  1st  Deployment,  (3)  After  1st  Deployment,  (4)  After  2nd  Deployment, 
(5)  After  1st  Sea  Tour,  and  (6)  After  2nd  Sea  Tour. 
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Knowledge  and  Performance  Tests 

Two  knowledge  tests  were  given  in  the  AAMC:  one  following  the  units  on  nuclear 
submarines  and  one  following  the  units  on  diesel  submarines.  The  knowledge  tests 
assessed  information  required  for  performing  the  scenario  exercises.  For  the  comparison 
between  AAMC  and  CCAA  courses,  trainees  in  both  courses  were  given  the  AAMC 
knowledge  test  on  the  nuclear  units.  For  the  pre-post  comparisons  for  the  AAMC, 
trainees  were  first  given  the  nuclear  and  diesel  tests  before  they  began  the  course  and  then 
were  given  the  same  tests  again  at  the  scheduled  time  during  the  course. 

Seven  scenario  exercises  were  given  as  performance  tests  in  the  AAMC.  Trainee 
performance  on  the  final  nuclear  and  diesel  scenarios  was  used  to  evaluate  accelerated 
expertise  on  ASW  mission  skills.  The  scenarios  were  designed  to  both  train  and  test  the 
mission  skills  listed  in  Table  1.  The  final  scenarios  for  both  nuclear  and  diesel 
submarines  were  weighted  most  heavily  for  final  course  grade.  These  scenarios  also 
placed  more  emphasis  on  skill  application  than  on  knowledge. 

Results 

CCAA  and  AAMC  Knowledge  Test  Performance 

A  t-test  compared  CCAA  and  AAMC  trainees  on  the  knowledge  test  for  the  nuclear 
units.  (Knowledge  test  data  for  the  diesel  units  and  performance  test  data  for  the  scenarios 
were  not  available  for  CCAA  trainees  because  the  course  was  discontinued.)  The  AAMC 
trainees  scored  significantly  higher  than  CCAA  trainees  (p  <  .01).  The  mean  score  for 
AAMC  trainees  was  69.27  ( SD  =  7.94)  and  the  mean  score  for  CCAA  trainees  was  53  43 
(SD  =  7.52). 

AAMC  Pretest  and  Posttest  Performance 

Two  tests  compared  the  pretest  and  posttest  performance  of  AAMC  trainees  on  the 
diesel  and  nuclear  knowledge  tests.  For  both  tests,  posttest  performance  was  significantly 
higher  than  pretest  performance  ip  <  .01).  Table  2  presents  the  means  and  standard 
deviations  for  these  analyses. 


Table  2 


Means  and  Standard  Deviations  for  the  Diesel  and 
Nuclear  Pretests  and  Posttests 


Knowledge  Test 

Pretest 

Posttest 

Diesel 

3.34 

89.66 

SD  =  2.87 

SD  =  6.71 

Nuclear 

13.76 

82.77 

SD  =  9.18 

SD  -  8.91 
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Mission  Skills  Survey 


For  the  mission  skills  survey,  a  statistical  consensus  was  determined  for  each  skill. 
This  was  done  by  calculating  the  skill’s  mean  rating  score  and  assigning  the  skill  scale 
rating  closest  to  the  mean.  Table  3  displays  the  consensus  scale  rating  and  mean  rating 
score  for  each  skill.  Table  2  shows  that  all  skills  received  the  highest  mission  importance 
consensus  rating. 


Table  3 


Consensus  Ratings  for  AW  Experts  on  Mission  Importance  and  Time  to 
Proficiency  and  Mean  Rating  Scores  for  Each  Skill/Behavior 


Skill  or  Behavior 

Mission  Importance 
(No.  of  Raters  =  6) 

Time  to  Proficiency 
(No.  of  Raters  =  6) 

Call  initial  contact. 

Absolutely  required 

Mean  =  4.00 

After  1st  deployment 

Mean  =  2.66 

Select  appropriate  signature  components 
for  tracking. 

Absolutely  required 

Mean  =  4.00 

Before  1st  deployment 

Mean  =  2.33 

Record  time,  frequency,  sonobuoy,  and 
appropriate  notes  for  all  acoustic 
signature  components  of  interest. 

Absolutely  required 

Mean  =  3.83 

After  1st  deployment 

Mean  =  2.66 

Recognize  and  report  intermittent  and 
transient  acoustic  sources. 

Absolutely  required 

Mean  =  3.66 

After  1st  deployment 

Mean  =  3.33 

Recognize  and  report  speed-related 
components. 

Absolutely  required 

Mean  =  3.66 

After  1st  deployment 

Mean  =  3.33 

Recognize  and  report  closest  point  of 
approach,  from  weak  indicators  to 
obvious  ones. 

Absolutely  required 

Mean  =  4.00 

Before  1st  deployment 

Mean  =  2.00 

Calculate  speed  from  propulsion-related 
components. 

Absolutely  required 

Mean  =  4.00 

Before  lsl  deployment 

Mean  =  2.00 

Report  speed  changes  in  a  timely  manner. 

Absolutely  required 

Mean  =  4.00 

Before  Is'  deployment 

Mean  =  2.16 

Report  and  record  contact  and  major 
component  losses. 

Absolutely  required 

Mean  =  4.00 

Before  1st  deployment 

Mean  =  2.33 

Prepare  an  off-station  report,  including 
classification  of  target  of  interest  with 
justification  based  on  acoustic  signature. 

Absolutely  required 

Mean  =  3.83 

After  1st  deployment 

Mean  =  2.66 
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AAMC  Accelerated  Expertise 


To  evaluate  accelerated  expertise  for  AAMC  trainees,  trainee  performance  on  the 
final  nuclear  and  diesel  scenarios  was  scored  for  each  of  the  mission  skills  listed  in  Table 
1.  Course  instructors  scored  trainees  on  a  5  point  scale:  (1)  Not  Proficient,  (2)  Below 
Average  Proficiency,  (3)  Average  Proficiency,  (4)  Above  Average  Proficiency,  and  (5) 
Highly  Proficient.  Mean  scores  for  all  the  skills  were  three  or  better. 

The  scale  consensus  rating  for  time  to  proficiency  and  the  mean  performance  scores 
for  the  final  diesel  and  nuclear  scenarios  for  each  skill  are  displayed  in  Table  4.  The 
mean  performance  scores  for  trainees  on  skills  rated  as  full  proficiency  achieved  “Before 
lsl  Deployment”  was  3.66  for  the  nuclear  scenario  and  3.43  for  the  diesel  scenario.  The 
mean  performance  scores  for  trainees  on  skills  rated  as  full  proficiency  achieved  “After 
the  Is'  Deployment”  was  3.98  for  the  nuclear  scenario  and  3.82  for  the  diesel  scenario.” 
These  scores  show  above  average  proficiency  for  all  tasks  and  offer  clear  evidence  of 
accelerated  expertise.  That  is,  the  skill  level  of  average  AAMC  graduate  is  above  average 
on  tasks  that  are  normally  acquired  in  follow-on  training  and  after  the  1’  deployment. 

Table  4 


Nuclear  and  Diesel  Scenario  Skill  Scores  for  AAMC  Trainees 


Skill  or  Behavior 

Time  to  Proficiency 
Consensus  Rating 

Nuclear 

Score 

Diesel 

Score 

Call  initial  contact. 

After  lbt  Deployment 

4.66 

4.23 

Select  appropriate  signature  components  for  tracking. 

Before  1st  Deployment 

4.40 

4.43 

Record  time,  frequency,  sonobuoy,  and  appropriate 
notes  for  all  acoustic  signature  components  of 
interest. 

After  1“  Deployment 

3.74 

4.17 

Recognize  and  report  intermittent  and  transient 
acoustic  sources. 

After  lil  Deployment 

3.00 

3.00 

Recognize  and  report  speed-related  components. 

After  1st  Deployment 

3.94 

Recognize  and  report  closest  point  of  approach,  from 
weak  indicators  to  obvious  ones. 

Before  lsl  Deployment 

4.37 

m 

Calculate  speed  from  propulsion-related  components. 

Before  lsl  Deployment 

3.57 

Report  speed  changes  in  a  timely  manner. 

Before  1st  Deployment 

3.00 

3.00 

Report  and  record  contact  and  major  component 
losses. 

Before  1st  Deployment 

3.00 

3.00 

Prepare  an  off-station  report  including  classification 
of  target  of  interest  with  justification  based  on 
acoustic  signature. 

After  1st  Deployment 

4.54 

4.71 
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Conclusions 


Two  conclusions  can  be  drawn  from  this  study.  First,  the  IMAT  approach  to  training, 
which  is  based  on  computer-generated  dynamic  displays  (scientific  visualization)  and 
instructional  design  and  delivery  strategies  designed  to  enhance  cognition,  produced 
substantial  gains  in  subject  matter  knowledge.  Second,  performance  on  the  AAMC 
scenario  tasks  support  the  conclusion  that  AAMC  trainees  experienced  accelerated 
expertise.  This  finding  indicates  that  the  IMAT-based  AAMC  should  be  capable  of 
overcoming  the  lack  of  practice  opportunities  available  in  today’s  typical  fleet  operations. 
Overall,  the  results  show  that  IMAT  is  a  highly  effective  training  system  that  offers  a 
viable  solution  for  many  of  the  training  requirements  and  challenges  faced  in  the  post 
Cold  War  world. 
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